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INTEGRATED INPUT/OUTPUT CONTROLLER 
BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to concurrent 
processing and mapping of multiple input/output requests and 
more particularly to input/output controllers to interface with 
an array of electronic devices such as computers, servers, 
networks, storage devices and the like. 

2 . Background Information 

Low-level control of a single disk drive to store data on 
one or more spinning magnetic disks is well known. Usually a 
dedicated disk drive controller performs the low level control 
of moving the read/write heads to the appropriate track and 
sector to read, write, or format the spinning magnetic disks. 

High-level control of a single disk drive is typically 
provided by software such as a software driver that interfaces 
the disk drive to a particular operating system of a computer. 
Another type of high-level control of a single disk drive is 
partitioning software that partitions a single disk drive into 
multiple volumes of storage areas. 

Another type of high-level control for multiple disk drives 
is redundant array of individual disks (RAID) software that is 
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used to provide redundancy and increase reliability of the 
storage' of data when multiple disks are available. Different 
levels of RAID software are available including RAID 0, RAID 1 
and RAID 5, which provide different mechanisms of redundancy and 
reliability depending upon the system level requirements. 

As the amount of data to be stored has become greater and 
the number of users requiring access to data has become larger, 
it has become a greater challenge to manage data storage. To 
ease data storage management, such as data backup and disaster 
recovery, it has become desirable to centralize data storage to 
a common pool of disks. The centralized data storage can be made 
available to multiple users over a network. Furthermore with the 
advent of the internet, centralized data storage provides 
increased data availability to a broader and larger 
distribution of users. 

Because the amount of data has grown and the number of 
users requiring access to centralized data storage has 
increased, the high level control of multiple disk drives by 
software alone has become disadvantageous. A main disadvantage 
to high-level control by software is that it is slow and creates 
a performance bottleneck in the flow of data into and out of the 
multiple array of disk drives. 

It is desirable to overcome the disadvantages of the prior 

art . 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The features and advantages of the present invention will 
become apparent from the following detailed description of the 
present invention in which: 

Figure 1 is a system block diagram in which the present 
invention is utilized. 

Figure 2 is a functional block diagram of an embodiment of 
the present invention. 

Figure 3 is a process diagram of a Host Interface subsystem 
of the present invention illustrating input and output packets 
and tables utilized by a Host Exchange Controller of the Host 
Interface subsystem. 

Figure 4 is a process diagram for the Host Interface 
subsystem of the present invention illustrating input and output 
packets utilized by a Command Decode controller of the Host 
Interface subsystem. 

Figure 5 is a process diagram of a Cache Manager of the 
present invention illustrating the input and output packets and 
tables utilized by the Cache Manager. 

Figure 6 is a process diagram of a Disk Mapping Controller 
of the present invention illustrating the input and output 
packets utilized by the Disk Mapping Controller. 
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Figure 7 is a process diagram of a Disk Exchange Controller 
of the present invention illustrating the input and output 
packets utilized by the Disk Exchange Controller. 

Figure 8 is a block diagram illustrating fields of a Range 
Operation Request packet utilized in the present invention. 

Figure 9 is a command flow diagram showing a typical 
sequence of operation of a block read command. 

Figure 10 is a command flow diagram showing a typical 
sequence of operation of a block write command. 

Like reference numbers and designations in the drawings 
indicate like elements providing similar functionality. 
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BRIEF SUMMARY OF THE INVENTION 

The present invention is briefly summarized by the claims 
that follow below. 
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DETAILED DESCRIPTION 

In the following detailed description of the present 
invention, numerous specific details are set forth in order to 
provide a thorough understanding of the present invention. 
However, it will be obvious to one skilled in the art that the 
present invention may be practiced without these specific 
details. In other instances well known methods, procedures, 
components, and circuits have not been described in detail so as 
not to unnecessarily obscure aspects of the present invention. 

The integrated I/O controller of the present invention 

integrates host and peripheral connections (such as disk or 

tape) , media protocol controllers (such as Fiber channel, 

Ethernet or Inifinband protocol management) , caching and buffer 

management hardware, block request mapping algorithms, packet 

management utilities and an embedded microcontroller. The 

integrated I/O controller performs resource workload balancing 

by monitoring a set of operational parameters and routing block 

level requests to the most appropriate resource. For example, 

two disk drives may both be able to satisfy the current data 

request. The integrated I/O controller will dispatch the 

command to the drive most likely to respond to the request in 

the least amount of time based upon the monitored parameters. 

The integrated I/O controller also provides improved fault 
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tolerance through its integration and reliability as well as its 
layers of management. The integrated I/O controller provides 
high-level control of a plurality of disks using hardware 
circuits in order to increase control efficiency and processing 
speeds. This high level control of the plurality of disks is 
integrated into one semiconductor integrated circuit. High- 
level I/O commands from the server are parsed by the integrated 
I/O controller and then mapped and routed to the appropriate 
peripheral (disk, tape, optical drive) in the array of 
peripherals available to the server. The subsequent data is 
then routed to/from the Server from/to the designated peripheral 
by the integrated I/O controller. 

In a RAID implementation the integrated I/O controller 
makes the array of physical disk drives transparent to the 
server. The array of physical disk drives are grouped together 
to form larger Logical disk drives and are referenced by the 
Server as Volumes via a Logical Unit Number (LUN) . Algorithms 
for RAID mapping are executed by the integrated I/O controller 
for managing data accesses to the Volumes. The integrated I/O 
controller uses microcoded structures and state machines 
operating concurrently in much of its circuitry, as opposed to 
firmware and software that operates sequentially. The 
microcoded structures and state machines used in the present 
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d ata hloC y et to b e re qU este d »V - Server or «. ^ ^ 

to be written to the physical disks. 

+n Ficrure 1, a system block diagram of a 
Referring now to Figure 1, * 

- v mo is illustrated. The storage 
simple storage area network 100 is 

area network 100 is exemplary of a system in which the 

inte grated I/O controller of the present invention is utilized 

and can be a local area network (LAN) , a metropolitan area 

, a wide area network (WAN) - In the storage area 
network (MAN) or a wide area 

M 100, a client arra y ,0, o £ clients 101.101* »- - 
writ e aa t a to at least one dl s* array ,0, havin, a plurality ot 

resid es an x/0 — , «. -i* «» functionality to P rovi d e 

* ^ralized data storage management 
centralized data storage and centralized 

for the network 100. 

In „ te to access the plurality of 103, the clients 

to a hub/switch 106 via cabling or 
clients 101A-101N couple to a huo/ 

1 r\Q n f a server array 112 
105 One or more servers 108 of a serv 
connection lUo- uue 

4- thP network via cable or 
couple to the hub/switcn 106 as part of the 
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other type o £ networ* connection 107. In one e*odi m ent, the 
hub/ switch !06 is an Ethernet switch and the networ. 100 an 
Et hernet networK. The c.Un, or connection 10S is usually 

controller UO. Alternatively the one or .ore servers 108 can 
co uple throu g h cahlin, or connection 10, to a Ei b re Channel 
switch 1U ana then the I/O controller UO throu g h ca b Un g or 
hos t connection U, — in g either Eihre Channel, Ethernet or 
In£1 „i b and as the protocol «or —cation. When the switch 

I/0 controller 110, the ten, Ea b ric is use. to descri.e this 
s tora g e area networ. topolo g y. The I/O controller UO couples 
to the plurality o £ disKs 103 and associated dis* array 10, *y 
way o £ ca.lin. or connection 117. - caflin, or connection 117 
is usually Ei b re Channel titrated Loop, *ut can also b e 
parallel SCSI or InUnihand. The ca b lin g or connection 117 rs 

v, 'n a o the physical media connection 
also referred to herein as the pny 

th, of disk 103 maybe optical disks, 
interface 117. The plurality of disk 

m a g netic disKs, se^etic dis.s, tape drives or other data 
st ora 9 e devices. The clrents 101A-101* of the client array Kl 
in one e^odi m ent are personal computers b ut can be other 
electronic or electrical devices coupled into a networK 
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configuration for storing and accessing data from the plurality 
of disks 103. 

Referring now to Figure 2, a functional diagram of a 
printed circuit board (PCB) 200 containing an integrated I/O 
controller device 210 of the present invention is illustrated. 
The I/O controller system 110 includes the PCB 200. As 
illustrated in Figure 2, the PCB 200 includes the integrated I/O 
controller device 210, a host connector 201, at least one or 
more disk connectors 202, a cache buffer memory 204 a cache 
state table buffer memory 203, a system micro-processor 205, and 
program memory (such as FLASH, ROM, and SRAM) 206 for storing 
the microprocessor program, configuration tables and other 
program structures. The integrated I/O controller device 210, a 
single silicon integrated circuit, couples to the components of 
the printed circuit board 200 as shown in Figure 2. The host 
connector 201 is for coupling the I/O controller device 210 to 
one or more servers 108 of the server array 112 via the network 
switch 111. The plurality of disk connectors 202 are for 
coupling the integrated I/O controller device 210 to at least 
one disk array 102 having the plurality of disks 103. The cache 
buffer 204 provides temporary storage for all data blocks 
transferred to/from the plurality of Servers 112 and the 
plurality of disks 103. The cache state table buffer memory 203 
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>~ a r«v 112 and the plurality of 
of servers 108 of the server array 

disks 103 of the disk array 102. 

970 couples to the cache buffer memory 
The buffer manager 270 couples 

oin additionally couples to the disk 
204. The buffer manager 270 additiona y 
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interface subsystem 230 and the host interface subsystem 220 by 
way of data channels 212 for the transfer of data to/from the 
buffer memory 204 from/to a host server 108 or a disk 103 of the 
disk array. The buffer manager 270 uses traditional arbitration 
methods to arbitrate between multiple concurrent access requests 
from/to the host interface subsystem 220, disk interface 
subsystem 230 and micro controller subsystem 240 to/from the 
cache buffer memory 204. 

The cache manager 2 60 couples to the cache state table 
buffer 203 of the cache memory and manages the coherency of data 
blocks in the cache buffer memory 204. The cache memory for the 
cache buffer memory 204 and the cache state table buffer 203 may 
be a dynamic random access memory or other type of memory. 
Additionally, the cache buffer memory 204 and cache state table 
buffer 203 could be integrated into the integrated I/O 
controller device 210 but it is preferable that it be external 
in order to provide system flexibility. 

The host interface subsystem 220 of the integrated I/O 
controller device 210 includes a Host Exchange Controller (HEC) 
222 and a Command Decode Controller (CDC) 224. The host 
exchange controller 222 controls the physical connection and 
protocol between the one or more servers 108 of the server array 
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112 and the integrated I/O controller device 210. This host 
connection- or attachment 119 to the servers 108 can be Fibre 
Channel, Ethernet, PCI, PCI-X or Infiniband in alternate 
embodiments. The command decode controller 224 parses all 
commands received from the one or more servers 108 of the server 
array 112 and extracts the data block level read, write req uests 

-v " ' 

while routing all other commands to the micro controller 
subsystem 240. All commands over the host connection are 
preferably in the form of the Small Computer System Interface 
(SCSI) Command Descriptor Block (CDB) standard. 

The disk interface subsystem 230 controls the connections 
between the plurality of disks 103 of the disk array 102 and the 
integrated I/O controller device 210 and includes one or more 
Disk Exchange Controllers 232A-232N. A preferred embodiment of 
the disk interface subsystem executes the industry standard 
fibre channel connection protocol (FC-AL, FC-AL2 or FC-AL3) on a 
physical level. To send and receive commands and data with the 
plurality of disks 103, the integrated I/O controller device 210 
uses the SCSI over Fibre Channel protocol (such as FCP, FCP2 or 
SCSI-3) . 
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The micro-controller subsystem 240 handles non-data flow 
commands, error and exception handling events as well as system 
initialization . 

Referring now to Figure 3, a process diagram of the Host 
Exchange Controller (HEC) 222 is illustrated. The host exchange 
controller 222 receives command and data frames from the 
physical media interface (Fibre Channel, Ethernet, Infiniband) . 
Data frames received by the host exchange controller 222 are 
routed to the buffer manager 270 over the data channels 212. 
The command frames received by the host exchange controller 222, 
such as command frames 301 illustrated in Figure 3, are stored 
as command entries into a host exchange table 303. The host 
exchange table 303 in one embodiment can hold thousands of 
command entries . 

Each command entry 313 in the host exchange table 303 has a 
tag number 311 which is either assigned or implied. In one 
embodiment, the tag number 311 is the same as the entry number, 
(i.e. the physical position of the entry), in the host exchange 
table 303. The physical location of the command entry in the 
host exchange table 303 implies the tag number and thus an 
actual tag number field 311 need not be stored with each entry. 

Each entry 313 in the host exchange table also includes 
several other fields to properly track the progress of the 
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command through the integrated I/O controller device 210 and 
subsequently transfer data blocks to/from the requesting server 
108. Field 305 contains network context information about the 
command which includes the Servers address on the network so 
response data can be routed properly back to the requesting 
server. Field 307 contains the SCSI CDB. Field 309 contains 
buffer pointers to physical locations in the cache buffer 204 
where the data will be stored. The Cache Manager 260 assigns the 
proper values and stores them in field 309 of each entry 313 
when the buffer space is physically allocated. Field 311 
represents the unique tag number associated with this host 
exchange entry 313. 

After the host exchange controller 222 creates the command 
entry 313 for a received command packet 301, a tag packet 315 
containing the tag number for the received command is formed by 
the host exchange controller. The tag packet 315 is then sent 
to the Command Decode Controller 224 of host interface subsystem 
220. 

Referring now to Figure 4, a process diagram for the 
Command Decode Controller 224 is illustrated. The command decode 
controller 224 receives the tag packets 315 including the tag 
numbers pointing to SCSI CDBs' that need further processing. 
When its ready to process the next command, the command decode 
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controller 224 reads the command entry 313 associated with a tag 
packet 315, extracts the SCSI CDB from it, and validates the 
contents of the command entry. The command decode controller 224 
includes hardware circuitry and a microcoded state-machine that 
specializes in decoding and queuing dataflow requests, such as 
block read and write commands. All commands which are received 
by the command decode controller 224 and determined to be valid, 
are placed into a pool or table of command queues 403. A 
separate command queue 403A-403N is maintained in the pool or 
table of command queues 403 for each volume of the disk array 
112 that is accessible to the server array 112. The command 
decode controller 224 implements the industry defined SCSI 
Architectural Model (SAM) for command processing, including 
command ordering and queuing. If implemented as a RAID 
controller, the command decode controller 224 translates the 
Logical Block Address (LBA) of the requested volume into a 
Global (or generic) logical block address (GLBA) understood by 
other circuitry in the integrated I/O controller device 210. 
This GLBA along with a sector count from the SCSI CDB, the 
direction of the requested transfer (read or write) , and tag 
number 311 are combined together to form a Range Operation 
Request (Ror) 801 which is further described below and 
illustrated in Figure 8. The Ror 801 is combined with a flag 
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field 405 to form an output packet 401 from the command decode 
controller 224 which is sent to the Cache Manager 2 60. In one 
embodiment of the invention the flag field 405 includes 
information about the source of the packet as well as routing 
information defining the destination or recipient of the packet. 
The flag field 405 can be expanded to include additional 
information about the process state of the command entry 313 to 
assist the micro-controller subsystem 240 in the event of an 
error condition or in the development and debug of the 
integrated I/O controller device 210. Any task management 
function required by an incoming host command, such as a request 
for the capabilities and configuration of the integrated I/O 
controller via SCSI Mode pages , is routed by the command decode 
controller 224 to the micro-controller subsystem 240. 

Referring now to Figure 8, the format of the Range 
Operation request (Ror) packet 801 is illustrated. The Ror 
packet 801 is used to communicate command requests between 
functional blocks of the integrated I/O controller device 210. 
The Ror packet 801 includes a field 803, a field 805, a field 
807 and a field 809. Field 803 contains the starting GLBA for 
the range of blocks to be operated upon. Field 805 of the Ror 
801 contains the number of blocks to be operated upon (range) 
which depends upon if the Ror is generated by the cache manager 
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260, the command decode controller 224 or another functional 
blocks of the integrated I/O controller device 210. If the 
command is generated by the command decode controller 224, the 
value of the range (i.e. number of blocks to be operated upon) 
is generated from the sector count of the SCSI CDB. If the 
command is generated by the Cache Manager 260, the value of the 
range (i.e. number of blocks to be operated upon) represents the 
number of blocks in the cache buffer 204 to which access is 
desired in order to read or write data blocks to/from the disk 
array 102 and the plurality of drives 103. Field 807 indicates 
the type of range operation being requested, either a Read data 
operation or a Write data operation. Field 809 contains the tag 
number 311 from which the given Ror 801 is derived or 
associated. 

Referring now to Figure 5, a process diagram for the Cache 
Manager 2 60 is illustrated. The cache manager 260 manages the 
state of all blocks within the cache buffer memory 204. The 
cache manager receives an input packet 401 including an Ror 
packet 801 and flags 405. The Ror 801 is extracted from the 
input packet 401 by the cache manager 2 60 and the range field 
805 is read. The cache state buffer 203 includes a cache state 
table 501 which has entries each being associated with the data 
blocks stored into the cache buffer 204. The cache manager 260 
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to store this linked list of drive commands 615 as illustrated 
in Figure 6. In case of the linked list of drive command 615, 
the disk mapping controller 250 sends a chained command packet 
609 which includes a chained command pointer 611 and the tag 
number 311 to the disk exchange controllers 232A-232N. The 
chained command pointer 611 points to the list of drive commands 
615 in the memory for processing by the Disk Exchange 
Controllers 232A-232N. 

Figure 7 is a process diagram for each of the Disk Exchange 
Controllers (DEC) 232A-232N. Each disk exchange controller 232A- 
232N receives either a basic command packet 601 or alternatively 
a chained command packet 609 if chained commands are used. The 
basic command packet 601 includes the tag number 311 and a basic 
drive command 603. The chained command packet 609 includes the 
tag number 311 and the chained command pointer 611. If a disk 
exchange controller 232A-232N receives the basic command packet 
601, it converts the basic drive commands 603 into SCSI command 
descriptor blocks (CDB) and then encapsulates them into an 
appropriate frame or output packet format 701 for transmission 
over the physical media connection interface 117 to the disks 
103 of the disk array 102. If a disk exchange controller 232A- 
232N receives the chained command packet 609, it reads the 
chained command pointer accesses a stored command in a chain and 
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sequence of operations for P 

stored in th e h os t — t a bl e 303. Th e h os t 

, ontroller 224 which indicates where the 
to the command decode controller 

«. ,» - » — - - - — 

nor 724 extracts and processes the SCSI 
Command Decode Controller 224 

, ■ ^ description of Figure 4. 
CDB 307 as previously explained m the 

, nsrltPt 401 to 

n^T- ??4 then passes a pacKet 
The command decode controller 224 

r 260 The cache manager 260 operates on the 
the Cache Manager 260. me 

~~ - - — — d - the description : in 

204 1S requ es t e. f ron\ Di s, — -.one, ,50 » - 

, o£a „c*e t5 03 — g . ^ 801 . <U.X 

generation of a new p 

— g — »0 operates c n ^ S03 as P^s, 
JLi- * U. — o £ TO e a. ^,3 U. ^ st s 
or «, one o £ «e — s V 
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* » 

_„ d ^ «... «. - — — ier 232R - 232 ; 

receiving the co^and packet acts as a SCSI initiator and 
P l u ralit y of d.sk -es i03 ana then — — » 

102 the cache Manager 260 updates the state 
of the disk array 102, tne v- 

r. „„ state Table 501 . The cache manager 
of the blocks in the Cache State 

n^r- 9?? that data is 
.the host exchange controller 222 tna 
260 then notifies the host 

th nacket 505. Upon receipt 
• „ • thP cache by sending the valid packet 
valid m the cacne 

v , <S05 the host exchange controller 222 
of the valid packet 505, the n 

fpr of data from the location specified in 
initiates the transfer of _ 

^09 listed in tne 

, ahle 303 for this command, to 
entry 313 in the host exchange table 303 

rpdin the network context information field 
the server specified in tne n 

305 in the same entry 313. 

in a flow diagram showing the 
Referring now to Figure 10, a 

•„« * host write command is 
^ Ari , f or processing a no si. 
sequence of operations for p 

j narket 301, including the host 
illustrated. A host command packet 

nd is received by the Host exchange controller 
write command, is recex 

, rnller 224 which indicates where the 
to the command decode controller 224 
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i „ rnrP qses the SCbi 
extracts and processes 

. „ f Figure 4. The command decode controller 
in the description o£ Figure 

, oacket 40! including Ror 801 to the Cache 
224 then passes a packet 

r ,60 The cache manager 260 examines the Ror 
MaM9 h buffer or storage space to 

oacket 401 to determine how much buffer 

. the cache buffer 204 for processing the data. Once 

3UOCate ^ ha s been allocated to process the Ror 301, 

enough buffer space has 

v. vn The cache manager 260 then 
* the tag number 311. ine 
ref erenced by the suf f icie nt buffer space 

nds a valid packet 505 indicate that suffx 
sends a vaixo. v caching is 

i 4- ^ fmm the host. It wriue 
■ nah le to receive data from 

" 3 k of data will be mediately processed for 

disabled, these blocks of data 

transfer to the plurality of disk dri.es 103 

102 . Th e write operation proceeds by generating a new P ac 

roller 250 indicating the range of data 
to the Disk Mapping Controller 

. the cache buffer 204. The disk mapping 
to be written from the cache 

... nac ket 503 as previously 
frnl1p , 250 operates on the packer 
controller V requests 

, • the description of Figure 6 and passes 
explained in the descr p 232A -232N 
to writ, data to one of the D.sk change Controllers 

nd packets 601 or the alternative chaxned 
by sending basic command packets 

ket 609 The disk exchange controller 232.-232N 
command packet 609. 
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• ♦ 

receive the con«and acts as a SCSX initiate, an, 

M „ ag es the sendin, o £ one or *ore disK co»ands to the 
pl uralit y o f dis* drives ,03 and then s„en« y the 

« MIS! ^.W-n an data has Oeen transferred to t h e 

4- of the blocks in the Cache State 
Manager 260 updates the state of the 

Table 501. 

The present invention has .an, advanta g es over the prior 
art one advantage of the present invention is that data 
com unication rates are increased t, in^ratino. t h e X/0 
con troUer rn hardware. another advantage o t t h e present 

>^t- i s also increased by providing 
invention is that data throughput is 

ci- -Ml another advantage of 
RAID mapping control in hardware. Still 

• the RAID mapping is flexible 

the present invention is that the rai 

through programmability 

♦-c havp been described and 
whil e certain exer»plary errfcodrments have 

■ o it- is to be understood that 
shown in the accompanying drawings, it is 

i ■ -| lust rative of and not restrictive 
such embodiments are merely illustrate 

on th e froad invention, and that this invention not b e ii m ited 
t0 the specific constructions and arran^ents shown and 
d escri>ed, since various other .educations m a y occur to those 

, ■ ,hP art For example, the integrated I/O 
ordinarily skilled xn the art. 
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controller ha s b ee„ as int e tfaCi n g «, a a«a Y o £ 

^O t ^«— ,/OMUOL t . on 

■n hardware, firmware, middleware, 
it-c: features in hardware, 
or some of its reduu 

are provid e d i. a process., «a d a bl e stora.e — as a 

mag ne t ic, optic.1. « se—uctor storage m e d iu m . 
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